{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tvgnzT1CKxrO"
   },
   "source": [
    "# Building a Model Experimentation Lineage with Pre-Built Code\n",
    "\n",
    "## Learning objectives\n",
    "\n",
    "1. Create a Cloud Storage bucket.\n",
    "2. Model experimentation with pre-built data pre-processing code.\n",
    "3. Visualize Experiment Lineage.\n",
    "\n",
    "\n",
    "## Introduction \n",
    "\n",
    "In this notebook, you learn how to integrate preprocessing code in a Vertex AI experiments. Also you will build the experiment lineage lets you record, analyze, debug, and audit metadata and artifacts produced along your ML journey.\n",
    "\n",
    "### Dataset\n",
    "\n",
    "This dataset is the UCI News Aggregator Data Set which contains 422,937 news collected between March 10th, 2014 and August 10th, 2014. Below are example records from the dataset:\n",
    "\n",
    "\n",
    "|ID |TITLE                                                                |URL                                                                                                                        |PUBLISHER        |CATEGORY|STORY                        |HOSTNAME           |TIMESTAMP    |\n",
    "|---|---------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|-----------------|--------|-----------------------------|-------------------|-------------|\n",
    "|1  |Fed official says weak data caused by weather, should not slow taper |http://www.latimes.com/business/money/la-fi-mo-federal-reserve-plosser-stimulus-economy-20140310,0,1312750.story\\?track=rss|Los Angeles Times|b       |ddUyU0VZz0BRneMioxUPQVP6sIxvM|www.latimes.com    |1394470370698|\n",
    "|2  |Fed's Charles Plosser sees high bar for change in pace of tapering   |http://www.livemint.com/Politics/H2EvwJSK2VE6OF7iK1g3PP/Feds-Charles-Plosser-sees-high-bar-for-change-in-pace-of-ta.html   |Livemint         |b       |ddUyU0VZz0BRneMioxUPQVP6sIxvM|www.livemint.com   |1394470371207|\n",
    "|3  |US open: Stocks fall after Fed official hints at accelerated tapering|http://www.ifamagazine.com/news/us-open-stocks-fall-after-fed-official-hints-at-accelerated-tapering-294436                |IFA Magazine     |b       |ddUyU0VZz0BRneMioxUPQVP6sIxvM|www.ifamagazine.com|1394470371550|\n",
    "|4  |Fed risks falling 'behind the curve', Charles Plosser says           |http://www.ifamagazine.com/news/fed-risks-falling-behind-the-curve-charles-plosser-says-294430                             |IFA Magazine     |b       |ddUyU0VZz0BRneMioxUPQVP6sIxvM|www.ifamagazine.com|1394470371793|\n",
    "|5  |Fed's Plosser: Nasty Weather Has Curbed Job Growth                   |http://www.moneynews.com/Economy/federal-reserve-charles-plosser-weather-job-growth/2014/03/10/id/557011                   |Moneynews        |b       |ddUyU0VZz0BRneMioxUPQVP6sIxvM|www.moneynews.com  |1394470372027|\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "Learn about [Vertex AI\n",
    "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
    "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
    "Calculator](https://cloud.google.com/products/calculator/)\n",
    "to generate a cost estimate based on your projected usage.\n",
    "\n",
    "Each learning objective will correspond to a __#TODO__ in this student lab notebook -- try to complete this notebook first before reviewing this [solution notebook](../solutions/build_model_experimentation_lineage_with_prebuild_code.ipynb).\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "i7EUnXsZhAGF"
   },
   "source": [
    "### Install additional packages\n",
    "\n",
    "Install additional package dependencies not installed in your notebook environment, such as TensorFlow or Vertex AI SDK. Use the latest major GA version of each package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "2b4ef9b72d43"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33m  WARNING: The scripts pyrsa-decrypt, pyrsa-encrypt, pyrsa-keygen, pyrsa-priv2pub, pyrsa-sign and pyrsa-verify are installed in '/home/jupyter/.local/bin' which is not on PATH.\n",
      "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\n",
      "\u001b[33m  WARNING: The script normalizer is installed in '/home/jupyter/.local/bin' which is not on PATH.\n",
      "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\n",
      "\u001b[33m  WARNING: The script tb-gcp-uploader is installed in '/home/jupyter/.local/bin' which is not on PATH.\n",
      "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\n",
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "tensorflow-io 0.18.0 requires tensorflow-io-gcs-filesystem==0.18.0, which is not installed.\n",
      "tfx-bsl 1.2.0 requires absl-py<0.13,>=0.9, but you have absl-py 0.13.0 which is incompatible.\n",
      "tfx-bsl 1.2.0 requires google-api-python-client<2,>=1.7.11, but you have google-api-python-client 2.15.0 which is incompatible.\n",
      "tfx-bsl 1.2.0 requires google-cloud-bigquery<2.21,>=1.28.0, but you have google-cloud-bigquery 2.34.4 which is incompatible.\n",
      "tfx-bsl 1.2.0 requires pyarrow<3,>=1, but you have pyarrow 5.0.0 which is incompatible.\n",
      "tensorflow 2.5.0 requires grpcio~=1.34.0, but you have grpcio 1.47.0 which is incompatible.\n",
      "tensorflow 2.5.0 requires six~=1.15.0, but you have six 1.16.0 which is incompatible.\n",
      "tensorflow 2.5.0 requires typing-extensions~=3.7.4, but you have typing-extensions 3.10.0.0 which is incompatible.\n",
      "tensorflow-transform 1.2.0 requires absl-py<0.13,>=0.9, but you have absl-py 0.13.0 which is incompatible.\n",
      "tensorflow-transform 1.2.0 requires google-cloud-bigquery<2.21,>=1.28.0, but you have google-cloud-bigquery 2.34.4 which is incompatible.\n",
      "tensorflow-transform 1.2.0 requires pyarrow<3,>=1, but you have pyarrow 5.0.0 which is incompatible.\n",
      "tensorflow-metadata 1.2.0 requires absl-py<0.13,>=0.9, but you have absl-py 0.13.0 which is incompatible.\n",
      "tensorboard 2.5.0 requires google-auth<2,>=1.6.3, but you have google-auth 2.11.0 which is incompatible.\n",
      "jupyterlab-git 0.11.0 requires nbdime<2.0.0,>=1.1.0, but you have nbdime 3.1.0 which is incompatible.\n",
      "httplib2 0.19.1 requires pyparsing<3,>=2.4.2, but you have pyparsing 3.0.9 which is incompatible.\n",
      "google-cloud-spanner 3.6.0 requires google-api-core[grpc]<2.0.0dev,>=1.22.2, but you have google-api-core 2.8.2 which is incompatible.\n",
      "google-cloud-spanner 3.6.0 requires google-cloud-core<2.0dev,>=1.4.1, but you have google-cloud-core 2.3.2 which is incompatible.\n",
      "google-cloud-dlp 1.0.0 requires google-api-core[grpc]<2.0.0dev,>=1.14.0, but you have google-api-core 2.8.2 which is incompatible.\n",
      "cloud-tpu-client 0.10 requires google-api-python-client==1.8.0, but you have google-api-python-client 2.15.0 which is incompatible.\n",
      "apache-beam 2.31.0 requires dill<0.3.2,>=0.3.1.1, but you have dill 0.3.4 which is incompatible.\n",
      "apache-beam 2.31.0 requires pyarrow<5.0.0,>=0.15.1, but you have pyarrow 5.0.0 which is incompatible.\n",
      "apache-beam 2.31.0 requires typing-extensions<3.8.0,>=3.7.0, but you have typing-extensions 3.10.0.0 which is incompatible.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "# The Vertex AI Workbench Notebook product has specific requirements\n",
    "IS_WORKBENCH_NOTEBOOK = os.getenv(\"DL_ANACONDA_HOME\")\n",
    "IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(\n",
    "    \"/opt/deeplearning/metadata/env_version\"\n",
    ")\n",
    "\n",
    "# Vertex AI Notebook requires dependencies to be installed with '--user'\n",
    "USER_FLAG = \"\"\n",
    "if IS_WORKBENCH_NOTEBOOK:\n",
    "    USER_FLAG = \"--user\"\n",
    "\n",
    "! pip3 install {USER_FLAG} --upgrade joblib fsspec gcsfs scikit-learn -q\n",
    "! pip3 install {USER_FLAG} --force-reinstall 'google-cloud-aiplatform>=1.15' -q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**: Please ignore the incompatibility warnings and errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hhq5zEbGg0XX"
   },
   "source": [
    "### Restart the kernel\n",
    "\n",
    "After you install the additional packages, you need to restart the notebook kernel so it can find the packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "EzrelQZ22IZj"
   },
   "outputs": [],
   "source": [
    "# Automatically restart kernel after installs\n",
    "import os\n",
    "\n",
    "if not os.getenv(\"IS_TESTING\"):\n",
    "    # Automatically restart kernel after installs\n",
    "    import IPython\n",
    "\n",
    "    app = IPython.Application.instance()\n",
    "    app.kernel.do_shutdown(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lWEdiXsJg0XY"
   },
   "source": [
    "## Before you begin"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WReHDGG5g0XY"
   },
   "source": [
    "#### Set your project ID\n",
    "\n",
    "**If you don't know your project ID**, you may be able to get your project ID using `gcloud`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "oM1iC_MfAts1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Project ID:  qwiklabs-gcp-04-c11d0df894dc\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "PROJECT_ID = \"qwiklabs-gcp-04-c11d0df894dc\"\n",
    "\n",
    "# Get your Google Cloud project ID from gcloud\n",
    "if not os.getenv(\"IS_TESTING\"):\n",
    "    shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n",
    "    PROJECT_ID = shell_output[0]\n",
    "    print(\"Project ID: \", PROJECT_ID)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qJYoRfYng0XZ"
   },
   "source": [
    "Otherwise, set your project ID here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "riG_qUokg0XZ"
   },
   "outputs": [],
   "source": [
    "if PROJECT_ID == \"\" or PROJECT_ID is None:\n",
    "    PROJECT_ID = \"qwiklabs-gcp-04-c11d0df894dc\"  # @param {type:\"string\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "wJyyYlKGYqaa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Updated property [core/project].\n"
     ]
    }
   ],
   "source": [
    "!gcloud config set project $PROJECT_ID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "88dd74c4c84e"
   },
   "source": [
    "#### Region\n",
    "\n",
    "You can also change the `REGION` variable, which is used for operations\n",
    "throughout the rest of this notebook.  Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.\n",
    "\n",
    "- Americas: `us-central1`\n",
    "- Europe: `europe-west4`\n",
    "- Asia Pacific: `asia-east1`\n",
    "\n",
    "You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.\n",
    "\n",
    "Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "5c615e53149f"
   },
   "outputs": [],
   "source": [
    "REGION = \"[your-region]\"  # @param {type: \"string\"}\n",
    "\n",
    "if REGION == \"[your-region]\":\n",
    "    REGION = \"us-central1\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "06571eb4063b"
   },
   "source": [
    "#### Timestamp\n",
    "\n",
    "If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append it onto the name of resources you create in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "697568e92bd6"
   },
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "PyQmSRbKA8r-"
   },
   "outputs": [],
   "source": [
    "# import necessary libraries\n",
    "\n",
    "import os\n",
    "import sys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zgPO1eR3CYjk"
   },
   "source": [
    "### Create a Cloud Storage bucket\n",
    "\n",
    "**The following steps are required, regardless of your notebook environment.**\n",
    "\n",
    "When you initialize the Vertex SDK for Python, you specify a Cloud Storage staging bucket. The staging bucket is where all the data associated with your dataset and model resources are retained across sessions.\n",
    "\n",
    "Set the name of your Cloud Storage bucket below. Bucket names must be globally unique across all Google Cloud projects, including those outside of your organization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "MzGDU7TWdts_"
   },
   "outputs": [],
   "source": [
    "BUCKET_NAME = \"qwiklabs-gcp-04-c11d0df894dc\"  # @param {type:\"string\"}\n",
    "BUCKET_URI = f\"gs://{BUCKET_NAME}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "cf221059d072"
   },
   "outputs": [],
   "source": [
    "if BUCKET_NAME == \"\" or BUCKET_NAME is None or BUCKET_NAME == \"[your-bucket-name]\":\n",
    "    BUCKET_NAME = PROJECT_ID + \"aip-\" + TIMESTAMP\n",
    "    BUCKET_URI = f\"gs://{BUCKET_NAME}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-EcIXiGsCePi"
   },
   "source": [
    "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "NIq7R4HZCfIc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating gs://qwiklabs-gcp-04-c11d0df894dc/...\n"
     ]
    }
   ],
   "source": [
    "! gcloud storage buckets create --location=$REGION --project=$PROJECT_ID $BUCKET_URI"   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ucvCsknMCims"
   },
   "source": [
    "Finally, validate access to your Cloud Storage bucket by examining its contents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "vhOb7YnwClBb"
   },
   "outputs": [],
   "source": [
    "! gcloud storage ls --all-versions --long $BUCKET_URI"   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0ljjTziu90rZ"
   },
   "source": [
    "### Set project folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "AARD6Fsr-DSi"
   },
   "outputs": [],
   "source": [
    "DATA_PATH = \"data\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "oigzvA4m937q"
   },
   "outputs": [],
   "source": [
    "!mkdir -m 777 -p {DATA_PATH}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UTjfRk2I9CGO"
   },
   "source": [
    "### Get the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "3V6W2nIo9FtL"
   },
   "outputs": [],
   "source": [
    "DATASET_URL = \"https://archive.ics.uci.edu/ml/machine-learning-databases/00359/NewsAggregatorDataset.zip\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "L4tPQ0lG9HoG"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2022-08-23 10:36:32--  https://archive.ics.uci.edu/ml/machine-learning-databases/00359/NewsAggregatorDataset.zip\n",
      "Resolving archive.ics.uci.edu (archive.ics.uci.edu)... 128.195.10.252\n",
      "Connecting to archive.ics.uci.edu (archive.ics.uci.edu)|128.195.10.252|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 29224203 (28M) [application/x-httpd-php]\n",
      "Saving to: ‘data/NewsAggregatorDataset.zip’\n",
      "\n",
      "NewsAggregatorDatas 100%[===================>]  27.87M  35.0MB/s    in 0.8s    \n",
      "\n",
      "2022-08-23 10:36:33 (35.0 MB/s) - ‘data/NewsAggregatorDataset.zip’ saved [29224203/29224203]\n",
      "\n",
      "Archive:  data/NewsAggregatorDataset.zip\n",
      "  inflating: data/temp/2pageSessions.csv  \n",
      "   creating: data/temp/__MACOSX/\n",
      "  inflating: data/temp/__MACOSX/._2pageSessions.csv  \n",
      "  inflating: data/temp/newsCorpora.csv  \n",
      "  inflating: data/temp/__MACOSX/._newsCorpora.csv  \n",
      "  inflating: data/temp/readme.txt    \n",
      "  inflating: data/temp/__MACOSX/._readme.txt  \n"
     ]
    }
   ],
   "source": [
    "!wget --no-parent {DATASET_URL} --directory-prefix={DATA_PATH}\n",
    "!mkdir -m 777 -p {DATA_PATH}/temp {DATA_PATH}/raw\n",
    "!unzip {DATA_PATH}/*.zip -d {DATA_PATH}/temp\n",
    "!mv {DATA_PATH}/temp/*.csv {DATA_PATH}/raw && rm -Rf {DATA_PATH}/temp && rm -f {DATA_PATH}/*.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "N-chqqLJAOpM"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying file://data/raw/newsCorpora.csv [Content-Type=text/csv]...\n",
      "Copying file://data/raw/2pageSessions.csv [Content-Type=text/csv]...            \n",
      "- [2/2 files][100.5 MiB/100.5 MiB] 100% Done                                    \n",
      "Operation completed over 2 objects/100.5 MiB.                                    \n"
     ]
    }
   ],
   "source": [
    "!gcloud storage cp --recursive {DATA_PATH}/raw $BUCKET_URI/{DATA_PATH}/raw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XoEqT2Y4DJmf"
   },
   "source": [
    "### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "pRUOFELefqf1"
   },
   "outputs": [],
   "source": [
    "# General\n",
    "import logging\n",
    "\n",
    "logger = logging.getLogger(\"logger\")\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "\n",
    "import collections\n",
    "import tempfile\n",
    "import time\n",
    "from json import dumps\n",
    "\n",
    "collections.Iterable = collections.abc.Iterable\n",
    "\n",
    "# Vertex AI\n",
    "from google.cloud import aiplatform as vertex_ai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oUiMepZvRiaa"
   },
   "source": [
    "### Define constants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "9zDMnU3gRhSn"
   },
   "outputs": [],
   "source": [
    "# Base\n",
    "DATASET_NAME = \"news_corpora\"\n",
    "DATASET_URI = f\"{BUCKET_URI}/{DATA_PATH}/raw/newsCorpora.csv\"\n",
    "\n",
    "# Experiments\n",
    "TASK = \"classification\"\n",
    "MODEL_TYPE = \"naivebayes\"\n",
    "EXPERIMENT_NAME = f\"{TASK}-{MODEL_TYPE}-{TIMESTAMP}\"\n",
    "EXPERIMENT_RUN_NAME = \"run-1\"\n",
    "\n",
    "# Preprocessing\n",
    "PREPROCESSED_DATASET_NAME = f\"preprocessed_{DATASET_NAME}\"\n",
    "PREPROCESS_EXECUTION_NAME = \"preprocess\"\n",
    "COLUMN_NAMES = [\n",
    "    \"id\",\n",
    "    \"title\",\n",
    "    \"url\",\n",
    "    \"publisher\",\n",
    "    \"category\",\n",
    "    \"story\",\n",
    "    \"hostname\",\n",
    "    \"timestamp\",\n",
    "]\n",
    "DELIMITER = \"\t\"\n",
    "INDEX_COL = 0\n",
    "PREPROCESSED_DATASET_URI = (\n",
    "    f\"{BUCKET_URI}/{DATA_PATH}/preprocess/{PREPROCESSED_DATASET_NAME}.csv\"\n",
    ")\n",
    "\n",
    "# Training\n",
    "TRAIN_EXECUTION_NAME = \"train\"\n",
    "TARGET = \"category\"\n",
    "FEATURES = \"title\"\n",
    "TEST_SIZE = 0.2\n",
    "SEED = 8\n",
    "TRAINED_MODEL_URI = f\"{BUCKET_URI}/deliverables/{TIMESTAMP}\"\n",
    "MODEL_NAME = f\"{EXPERIMENT_NAME}-model\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "inR70nh38PeK"
   },
   "source": [
    "### Initialize Vertex AI SDK for Python\n",
    "\n",
    "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "Nz0nasrh8T3c"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:MetadataStore default not found.\n"
     ]
    }
   ],
   "source": [
    "# Initialize the vertex AI\n",
    "# TODO: Your code goes here (\n",
    "    project=PROJECT_ID, experiment=EXPERIMENT_NAME, staging_bucket=BUCKET_URI\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "container:training,prediction,xgboost"
   },
   "source": [
    "### Set pre-built containers\n",
    "\n",
    "Set the pre-built Docker container image for training and prediction.\n",
    "\n",
    "\n",
    "For the latest list, see [Pre-built containers for training](https://cloud.google.com/ai-platform-unified/docs/training/pre-built-containers).\n",
    "\n",
    "\n",
    "For the latest list, see [Pre-built containers for prediction](https://cloud.google.com/ai-platform-unified/docs/predictions/pre-built-containers)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "XujRA5ueox9U"
   },
   "outputs": [],
   "source": [
    "SERVE_IMAGE = vertex_ai.helpers.get_prebuilt_prediction_container_uri(\n",
    "    framework=\"sklearn\", framework_version=\"1.0\", accelerator=\"cpu\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9JSL9CzsZZZT"
   },
   "source": [
    "### Initialize Experiment run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "AmHX6qy6ZVoq"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Associating projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532-run-1 to Experiment: classification-naivebayes-20220823103532\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.metadata.experiment_resources:Associating projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532-run-1 to Experiment: classification-naivebayes-20220823103532\n"
     ]
    }
   ],
   "source": [
    "run = vertex_ai.start_run(EXPERIMENT_RUN_NAME)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tysB_Zy68KIe"
   },
   "source": [
    "## Model experimentation with pre-built data pre-processing code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vEyXGFqq8cR_"
   },
   "source": [
    "### Data preprocessing\n",
    "\n",
    "In this step, you simulate some preprocessing on raw data to create the training dataset.\n",
    "\n",
    "Indeed it would happen that you have some data preprocessing that others within your team may have written. For that reason, you need a way to integrate the preprocessing code in your experiment runs to standardize and reuse all the complex data wrangling that you are dealing with. \n",
    "\n",
    "Using Vertex AI Experiments, you can track that code as part of the run execution by adding one `with` statement in your experiment context. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "43K9jkygpv9w"
   },
   "source": [
    "#### Create a Dataset Metadata Artifact\n",
    "\n",
    "First you create the Dataset artifact to track the dataset resource in the Vertex AI ML Metadata and create the experiment lineage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "configured-fetish"
   },
   "outputs": [],
   "source": [
    "# Use the vertex AI artifact create function\n",
    "raw_dataset_artifact = # TODO: Your code goew here (\n",
    "    schema_title=\"system.Dataset\", display_name=DATASET_NAME, uri=DATASET_URI\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZrtoR-V0GVzy"
   },
   "source": [
    "#### Create a preprocess module\n",
    "\n",
    "Next, you build a simple preprocessing module to convert the text case and remove the punctuation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "DjCc1Us_Gb5Q"
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Preprocess module\n",
    "\"\"\"\n",
    "\n",
    "import string\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "\n",
    "def preprocess(df: pd.DataFrame, text_col: str) -> pd.DataFrame:\n",
    "    \"\"\"\n",
    "    Preprocess text\n",
    "    Args:\n",
    "        df: The DataFrame to preprocesss\n",
    "        text_col: The text column name\n",
    "    Returns:\n",
    "        preprocessed_df: The datafrane with text in lowercase and without punctuation\n",
    "    \"\"\"\n",
    "    preprocessed_df = df.copy()\n",
    "    preprocessed_df[text_col] = preprocessed_df[text_col].apply(lambda x: x.lower())\n",
    "    preprocessed_df[text_col] = preprocessed_df[text_col].apply(\n",
    "        lambda x: x.translate(str.maketrans(\"\", \"\", string.punctuation))\n",
    "    )\n",
    "    return preprocessed_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vSVFPWU9DFw_"
   },
   "source": [
    "#### Add the `preprocessing` Execution\n",
    "\n",
    "You add the preprocessing step to track its execution in the lineage associated to Vertex AI Experiment. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "NI5e4a44EH4S"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:Start preprocess execution.\n",
      "INFO:root:Log preprocessing params.\n",
      "INFO:root:Preprocessing.\n",
      "INFO:root:Log preprocessing metrics and preprocessed_news_corpora dataset.\n"
     ]
    }
   ],
   "source": [
    "# Use the vertex AI start execution function\n",
    "with # TODO: Your code goes here (\n",
    "    schema_title=\"system.ContainerExecution\", display_name=PREPROCESS_EXECUTION_NAME\n",
    ") as exc:\n",
    "    logging.info(f\"Start {PREPROCESS_EXECUTION_NAME} execution.\")\n",
    "    exc.assign_input_artifacts([raw_dataset_artifact])\n",
    "\n",
    "    # Log preprocessing params --------------------------------------------------\n",
    "    logging.info(\"Log preprocessing params.\")\n",
    "    vertex_ai.log_params(\n",
    "        {\n",
    "            \"delimiter\": DELIMITER,\n",
    "            \"features\": dumps(COLUMN_NAMES),\n",
    "            \"index_col\": INDEX_COL,\n",
    "        }\n",
    "    )\n",
    "\n",
    "    # Preprocessing ------------------------------------------------------------\n",
    "    logging.info(\"Preprocessing.\")\n",
    "    raw_df = pd.read_csv(\n",
    "        raw_dataset_artifact.uri,\n",
    "        delimiter=DELIMITER,\n",
    "        names=COLUMN_NAMES,\n",
    "        index_col=INDEX_COL,\n",
    "    )\n",
    "    preprocessed_df = preprocess(raw_df, \"title\")\n",
    "    preprocessed_df.to_csv(PREPROCESSED_DATASET_URI, sep=\",\")\n",
    "\n",
    "    # Log preprocessing metrics and store dataset artifact ---------------------\n",
    "    logging.info(f\"Log preprocessing metrics and {PREPROCESSED_DATASET_NAME} dataset.\")\n",
    "    vertex_ai.log_metrics(\n",
    "        {\n",
    "            \"n_records\": preprocessed_df.shape[0],\n",
    "            \"n_columns\": preprocessed_df.shape[1],\n",
    "        },\n",
    "    )\n",
    "\n",
    "    preprocessed_dataset_metadata = vertex_ai.Artifact.create(\n",
    "        schema_title=\"system.Dataset\",\n",
    "        display_name=PREPROCESSED_DATASET_NAME,\n",
    "        uri=PREPROCESSED_DATASET_URI,\n",
    "    )\n",
    "    exc.assign_output_artifacts([preprocessed_dataset_metadata])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "blqKeaw9cigK"
   },
   "source": [
    "### Model Training\n",
    "\n",
    "In this step, you train a multinomial Naive Bayes pipeline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9DIhphAwcpl2"
   },
   "source": [
    "#### Create model training module\n",
    "\n",
    "Below the training module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "XAV2Ew1Gc-dk"
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Train module\n",
    "\"\"\"\n",
    "\n",
    "import joblib\n",
    "import pandas as pd\n",
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer\n",
    "from sklearn.metrics import accuracy_score, precision_score, recall_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "\n",
    "def get_training_split(\n",
    "    x: pd.DataFrame, y: pd.Series, test_size: float, random_state: int\n",
    ") -> (pd.DataFrame, pd.Series, pd.DataFrame, pd.Series):\n",
    "    \"\"\"\n",
    "    Splits data into training and testing sets\n",
    "    Args:\n",
    "        x: The data to be split\n",
    "        y: The labels to be split\n",
    "        test_size: The proportion of the data to be reserved for testing\n",
    "        random_state: The seed used by the random number generator\n",
    "    Returns:\n",
    "        x_train: The training data\n",
    "        x_test: The testing data\n",
    "        y_train: The training labels\n",
    "        y_test: The testing labels\n",
    "    \"\"\"\n",
    "\n",
    "    x_train, x_val, y_train, y_val = train_test_split(\n",
    "        x, y, test_size=test_size, random_state=random_state\n",
    "    )\n",
    "    return x_train, x_val, y_train, y_val\n",
    "\n",
    "\n",
    "def get_pipeline():\n",
    "    \"\"\"\n",
    "    Get the model\n",
    "    Args:\n",
    "        None\n",
    "    Returns:\n",
    "        model: The model\n",
    "    \"\"\"\n",
    "    model = Pipeline(\n",
    "        [\n",
    "            (\"vect\", CountVectorizer()),\n",
    "            (\"tfidf\", TfidfTransformer()),\n",
    "            (\"clf\", MultinomialNB()),\n",
    "        ]\n",
    "    )\n",
    "    return model\n",
    "\n",
    "\n",
    "def train_pipeline(model: Pipeline, X_train: pd.Series, y_train: pd.Series) -> Pipeline:\n",
    "    \"\"\"\n",
    "    Train the model\n",
    "    Args:\n",
    "        model: The model to train\n",
    "        X_train: The training data\n",
    "        y_train: The training labels\n",
    "    Returns:\n",
    "        model: The trained model\n",
    "    \"\"\"\n",
    "    model.fit(X_train, y_train)\n",
    "    return model\n",
    "\n",
    "\n",
    "def evaluate_model(model: Pipeline, X_test: pd.Series, y_test: pd.Series) -> float:\n",
    "    \"\"\"\n",
    "    Evaluate the model\n",
    "    Args:\n",
    "        model: The model to evaluate\n",
    "        X_test: The testing data\n",
    "        y_test: The testing labels\n",
    "    Returns:\n",
    "        score: The accuracy of the model\n",
    "    \"\"\"\n",
    "    # Evaluate model\n",
    "    y_pred = model.predict(X_test)\n",
    "\n",
    "    # Store evaluation metrics\n",
    "    # Store evaluation metrics\n",
    "    metrics = {\n",
    "        \"accuracy\": round(accuracy_score(y_test, y_pred), 5),\n",
    "        \"precision\": round(precision_score(y_test, y_pred, average=\"weighted\"), 5),\n",
    "        \"recall\": round(recall_score(y_test, y_pred, average=\"weighted\"), 5),\n",
    "    }\n",
    "    return metrics\n",
    "\n",
    "\n",
    "def save_model(model: Pipeline, save_path: str) -> int:\n",
    "    try:\n",
    "        with tempfile.NamedTemporaryFile() as tmp:\n",
    "            joblib.dump(trained_pipeline, filename=tmp.name)\n",
    "            ! gcloud storage cp {tmp.name} {save_path}/model.joblib\n",    "    except RuntimeError as error:\n",
    "        print(error)\n",
    "    return 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AHqNQ66EntqP"
   },
   "source": [
    "#### Add the `train` Execution\n",
    "\n",
    "Next, you add the training task to the experiment execution to update the experiment lineage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "5-kIK4obcbSr"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:root:Get training and testing data.\n",
      "INFO:root:Get model pipeline.\n",
      "INFO:root:Log data parameters.\n",
      "INFO:root:Log pipeline parameters.\n",
      "INFO:root:Train model pipeline.\n",
      "INFO:root:Evaluate model.\n",
      "INFO:root:Log training metrics.\n",
      "INFO:root:Generate prediction sample.\n",
      "INFO:root:Upload Model on Vertex AI.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "prediction sample: ['m' 'e' 'e' 't' 'e' 'b' 'b' 'e' 'm' 't']\n",
      "Copying file:///tmp/tmps2wwgyl6 [Content-Type=application/octet-stream]...\n",
      "/ [1 files][  6.3 MiB/  6.3 MiB]                                                \n",
      "Operation completed over 1 objects/6.3 MiB.                                      \n",
      "Creating Model\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.models:Creating Model\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Create Model backing LRO: projects/135696800439/locations/us-central1/models/3065944193588264960/operations/2276996565744746496\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.models:Create Model backing LRO: projects/135696800439/locations/us-central1/models/3065944193588264960/operations/2276996565744746496\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model created. Resource name: projects/135696800439/locations/us-central1/models/3065944193588264960@1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.models:Model created. Resource name: projects/135696800439/locations/us-central1/models/3065944193588264960@1\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To use this Model in another session:\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.models:To use this Model in another session:\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model = aiplatform.Model('projects/135696800439/locations/us-central1/models/3065944193588264960@1')\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.models:model = aiplatform.Model('projects/135696800439/locations/us-central1/models/3065944193588264960@1')\n"
     ]
    }
   ],
   "source": [
    "with vertex_ai.start_execution(\n",
    "    schema_title=\"system.ContainerExecution\", display_name=TRAIN_EXECUTION_NAME\n",
    ") as exc:\n",
    "\n",
    "    exc.assign_input_artifacts([preprocessed_dataset_metadata])\n",
    "\n",
    "    # Get training and testing data\n",
    "    logging.info(\"Get training and testing data.\")\n",
    "    x_train, x_val, y_train, y_val = get_training_split(\n",
    "        preprocessed_df[FEATURES],\n",
    "        preprocessed_df[TARGET],\n",
    "        test_size=TEST_SIZE,\n",
    "        random_state=SEED,\n",
    "    )\n",
    "    # Get model pipeline\n",
    "    logging.info(\"Get model pipeline.\")\n",
    "    pipeline = get_pipeline()\n",
    "\n",
    "    # Log training param -------------------------------------------------------\n",
    "\n",
    "    # Log data parameters\n",
    "    logging.info(\"Log data parameters.\")\n",
    "    vertex_ai.log_params(\n",
    "        {\n",
    "            \"target\": TARGET,\n",
    "            \"features\": FEATURES,\n",
    "            \"test_size\": TEST_SIZE,\n",
    "            \"random_state\": SEED,\n",
    "        }\n",
    "    )\n",
    "\n",
    "    # Log pipeline parameters\n",
    "    logging.info(\"Log pipeline parameters.\")\n",
    "    vertex_ai.log_params(\n",
    "        {\n",
    "            \"pipeline_steps\": dumps(\n",
    "                {step[0]: str(step[1].__class__.__name__) for step in pipeline.steps}\n",
    "            )\n",
    "        }\n",
    "    )\n",
    "\n",
    "    # Training -----------------------------------------------------------------\n",
    "\n",
    "    # Train model pipeline\n",
    "    logging.info(\"Train model pipeline.\")\n",
    "    train_start = time.time()\n",
    "    trained_pipeline = train_pipeline(pipeline, x_train, y_train)\n",
    "    train_end = time.time()\n",
    "\n",
    "    # Evaluate model\n",
    "    logging.info(\"Evaluate model.\")\n",
    "    model_metrics = evaluate_model(trained_pipeline, x_val, y_val)\n",
    "\n",
    "    # Log training metrics and store model artifact ----------------------------\n",
    "\n",
    "    # Log training metrics\n",
    "    logging.info(\"Log training metrics.\")\n",
    "    vertex_ai.log_metrics(model_metrics)\n",
    "\n",
    "    # Generate first ten predictions\n",
    "    logging.info(\"Generate prediction sample.\")\n",
    "    prediction_sample = trained_pipeline.predict(x_val)[:10]\n",
    "    print(\"prediction sample:\", prediction_sample)\n",
    "\n",
    "    # Upload Model on Vertex AI\n",
    "    logging.info(\"Upload Model on Vertex AI.\")\n",
    "    loaded = save_model(trained_pipeline, TRAINED_MODEL_URI)\n",
    "    if loaded:\n",
    "        model = vertex_ai.Model.upload(\n",
    "            serving_container_image_uri=SERVE_IMAGE,\n",
    "            artifact_uri=TRAINED_MODEL_URI,\n",
    "            display_name=MODEL_NAME,\n",
    "        )\n",
    "\n",
    "    exc.assign_output_artifacts([model])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "ee96d6d8c0fa"
   },
   "outputs": [],
   "source": [
    "run.end_run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0KQbDqiRBV9x"
   },
   "source": [
    "### Visualize Experiment Lineage\n",
    "\n",
    "Below you will get the link to Vertex AI Metadata UI in the console that will show the experiment lineage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "4L_XvSllBeqC"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Open the following link: https://console.cloud.google.com/vertex-ai/locations/us-central1/metadata-stores/default/artifacts/f629492a-3095-4a9f-9600-ab74a38150c3?project=qwiklabs-gcp-04-c11d0df894dc\n"
     ]
    }
   ],
   "source": [
    "print(\"Open the following link:\", exc.get_output_artifacts()[0].lineage_console_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TpV-iwP9qw9c"
   },
   "source": [
    "## Cleaning up\n",
    "\n",
    "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
    "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
    "\n",
    "Otherwise, you can delete the individual resources you created in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "uGvJieN5BjPG"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Deleting Context : projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532-run-1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Deleting Context : projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532-run-1\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Delete Context  backing LRO: projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532-run-1/operations/7339042546909184000\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Delete Context  backing LRO: projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532-run-1/operations/7339042546909184000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context deleted. . Resource name: projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532-run-1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Context deleted. . Resource name: projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532-run-1\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Deleting Context : projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Deleting Context : projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Delete Context  backing LRO: projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532/operations/1552854015803392\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Delete Context  backing LRO: projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532/operations/1552854015803392\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context deleted. . Resource name: projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Context deleted. . Resource name: projects/135696800439/locations/us-central1/metadataStores/default/contexts/classification-naivebayes-20220823103532\n"
     ]
    }
   ],
   "source": [
    "# Delete experiment\n",
    "exp = vertex_ai.Experiment(EXPERIMENT_NAME)\n",
    "exp.delete()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "id": "gW8Ddbr8xaKp"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Deleting Model : projects/135696800439/locations/us-central1/models/3065944193588264960\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Deleting Model : projects/135696800439/locations/us-central1/models/3065944193588264960\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Delete Model  backing LRO: projects/135696800439/locations/us-central1/operations/6514883815100383232\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Delete Model  backing LRO: projects/135696800439/locations/us-central1/operations/6514883815100383232\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model deleted. . Resource name: projects/135696800439/locations/us-central1/models/3065944193588264960\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:google.cloud.aiplatform.base:Model deleted. . Resource name: projects/135696800439/locations/us-central1/models/3065944193588264960\n"
     ]
    }
   ],
   "source": [
    "# Delete model\n",
    "model_list = vertex_ai.Model.list(filter=f'display_name=\"{MODEL_NAME}\"')\n",
    "for model in model_list:\n",
    "    model.delete()\n",
    "\n",
    "# Delete dataset\n",
    "for dataset_name in [DATASET_NAME, PREPROCESSED_DATASET_NAME]:\n",
    "    dataset_list = vertex_ai.TabularDataset.list(\n",
    "        filter=f'display_name=\"{dataset_name}\"'\n",
    "    )\n",
    "    for dataset in dataset_list:\n",
    "        dataset.delete()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "sx_vKniMq9ZX"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Removing gs://qwiklabs-gcp-04-c11d0df894dc/data/preprocess/preprocessed_news_corpora.csv#1661251081564100...\n",
      "Removing gs://qwiklabs-gcp-04-c11d0df894dc/data/raw/2pageSessions.csv#1661251002815678...\n",
      "Removing gs://qwiklabs-gcp-04-c11d0df894dc/data/raw/newsCorpora.csv#1661251003714955...\n",
      "Removing gs://qwiklabs-gcp-04-c11d0df894dc/deliverables/20220823103532/model.joblib#1661251212765094...\n",
      "/ [4/4 objects] 100% Done                                                       \n",
      "Operation completed over 4 objects.                                              \n",
      "Removing gs://qwiklabs-gcp-04-c11d0df894dc/...\n"
     ]
    }
   ],
   "source": [
    "# Delete Cloud Storage objects that were created\n",
    "delete_bucket = True\n",
    "\n",
    "if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
    "    ! gcloud storage rm --recursive $BUCKET_URI"   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "3f00c455b930"
   },
   "outputs": [],
   "source": [
    "!rm -Rf {DATA_PATH}"
   ]
  }
 ],
 "metadata": {
  "environment": {
   "name": "tf2-cpu.2-5.m76",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/tf2-cpu.2-5:m76"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
